Skip to content

Curation: pin the Hitchens and Bierce comparison instructions (#307 P3b) - #308

Merged
holden merged 4 commits into
mainfrom
claude/307-persona-pins
Nov 4, 2026
Merged

holden merged 4 commits into
mainfrom
claude/307-persona-pins

Conversation

@holden

@holden holden commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

#307 P3b: pin the Hitchens and Bierce comparison instructions

#303 (C4) built only the seam: a persona's lens is read by slug through Agents.Persona and composed with v7's grounded body by Packet.grounded_with_lens/1. Before the Stage 1 batch (#307, under #305) can run arms H and B, the two instructions have to be pinned and named. This PR does that and nothing else.

What changes

What does not change

Tests

  • comparison_instructions_test.exs pins both shas. It checks that each instruction equals v7 with Voltaire's lens replaced by the dossier's lens, where the lens is read straight from the JSON (disclosure, newline, selection) and also through Persona. It checks that both equal grounded_with_lens/1 of that lens, and that the allowlists admit exactly the four comparison instructions. A persona arm must see exactly v7's user message, schema and evidence fingerprint, with a different system prompt and prompt sha.
  • Focused: 56 tests, 0 failures, partition _s1c.
  • Full mix precommit at the head e011ca6, partition _s1f, 05:03:45–05:11:53 UTC: 19 doctests, 3,402 tests, 0 failures.
  • Earlier full runs: at a1b2632, one real failure. packet_test.exs hard-codes the pinned list; fixed in f90494d. At e011ca6 two runs each had one failure that is not this change's. In _s1c, the migration test's scratch database devils_dictionary_test_s1c_cmp was left by a run I had stopped (already_up). In _s1f, HealthLiveTest's scorecard row X2 is a live search-latency measurement (p95 under 150 ms) and flipped under load, 22 vs 21; that test passed 3 of 3 runs alone.
  • CodeRabbit CLI: not run. The CLI says its Git provider access needs reauthorization (sign out and in again).
  • Independent exact-head reviews, posted below: f90494d (nothing REQUIRED; RECOMMENDED 1 applied as e011ca6, 2 and 3 recorded as follow-ups), then e011ca6 (nothing REQUIRED or RECOMMENDED).

Part of #307 (P3b), under #305.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added Hitchens and Bierce persona options for comparison batch dry runs, using their respective reviewed selection lenses.
    • Added setup guidance and a sample four-arm comparison run.
  • Tests

    • Expanded checks for persona instructions, comparison eligibility, and quotation handling.

holden and others added 4 commits November 4, 2026 05:38
`hitchens-whole-entry-v1` and `bierce-whole-entry-v1` are v7's grounded
body and lens preface with each reviewed dossier's lens (#302), read by
slug through `Agents.Persona` and compiled in, as `grounded_with_lens/1`
composes them: the policy sections are v7's byte for byte. Both are named
in `WholeEntry.comparison_instructions/0`, so the section output, the
contract (`selection-output-v4`) and the evidence guidance admit them; no
publication path does. Public summaries under
`priv/curation/agents/instructions/`; the runbook names their model
configs. No migration; the configuration CHECK and author guard are
untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… in locators (#302 review RECOMMENDED 1-2)

The exact-head review of #302 at 2919d1a (c6622581655) recommended both;
carried here so #302 merged at its reviewed head.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Full precommit at a1b2632 (partition _s1c) failed only here: the
#254 P7 list of pinned instruction versions is literal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…307 review RECOMMENDED 1)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@holden

holden commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Independent exact-head review of f90494d

4 November 2026, by a fresh subagent of the #307 Stage 1 executor that had not seen the author's reasoning; posted verbatim by the executor. It began at a1b2632 and was extended to f90494d when precommit caught the literal list in packet_test.exs.

Head: git rev-parse HEAD = f90494d9481fc887f6015d7b970033071b7e18e2. The worktree is clean. Three commits on f63775a: 5ff01aa, a1b2632 and f90494d.

What I read: the full git diff f63775a f90494d; issue #307 §3 and P3b; packet.ex lines 200–470; whole_entry.ex; agents/persona.ex; every reader in lib/ of Packet.instruction_versions() or WholeEntry.instructions() (configurations, profiles, provenance, runtime smoke, dd.curation.editor, dd.runtime, plus the compile-time lists in contract, section_output and evidence_guidance); the DB CHECK and author guard in migration 20261101170541; the dossier test; the runbook test; and #304's summary test (curator_live_test.exs:673).

Independent checks (scratch scripts only):

  • Shas: I rebuilt both instructions from the source AST and the dossier JSON, without mix. They give 3463bce9…abe9ab for Hitchens and b02355a9…57f0 for Bierce, exactly the literals in the test. The same method reproduces v7 as a3219a5c….
  • Summaries: I applied Curator pages: directory, profile and version pages (#299 M1–M7) #304's 8-word test, exempting only Voltaire's lens. Neither summary contains any 8-word run of its instruction. Bierce's longest shared run is 7 words. Both start with # .
  • Compile manifest: compile.elixir lists the hitchens and bierce v1/profile.json files as external resources of packet.ex, so editing either file recompiles Packet. Packet already called Persona at compile time, so no new compile-time edge is added.
  • Release build: the text is compiled into the beam. Nothing reads the source tree at runtime.
  • Test commit (a1b2632): both changes are real checks, not vacuous. The prejudice guard still asserts [{"bierce", "INDISCRETION"}]. The locator check runs on two in-copyright Hitchens sources (11 locators): none contains U+2018 today, and one would fail.
  • Literal lists in test/: only packet_test.exs:26, now fixed in sorted order, and comparison_instructions_test.exs:142 hard-code instruction lists.

Publication and admission paths: adding the two names does not open any path that is not already open to generic-whole-entry-v1.

  • coherent?/1 refuses them in both shapes.
  • editor_binding refuses them: configurations.ex:274 allows only generic-editor-v1/v2, and Persona.compatible? accepts only Voltaire.
  • The DB author guard accepts only Voltaire template versions.
  • Persona.prepare/3, the persona admission rule and publishing_instructions/0 are unchanged.
  • No migration, no config/*.exs change and no mix.lock change.

Tests (partition _s1d, PHX_SERVER unset, DB port 5434; nothing was recompiled):

  • The five focused files plus packet_test.exs: 70 tests, 0 failures.
  • persona_by_slug, whole_entry, contract, evidence_guidance, provenance, dd_runtime_compare, configurations, packet_v2_scope: 103 tests, 0 failures.
  • mix format --check-formatted: exit 0.
  • No forced compile (a full run shared the build); warnings-as-errors is covered by precommit, which got past compile at a1b2632.

REQUIRED

None.

RECOMMENDED

  1. priv/curation/agents/instructions/bierce-whole-entry-v1.md:6: "the quiet contrast over the loud one" drifts from the dossier, which prefers the exact, economical contrast. A word like "precise, economical" would describe it accurately without quoting it.
  2. lib/devils_dictionary/curation/profiles.ex:167,209: this predates the change. The generic editor's seed and admission accept any pinned instruction, so a generic-editor version naming either new instruction can be admitted. It still cannot be bound (configurations.ex:274). Separately, the DB CHECK's ≤3 branch (20261101170541…:8) excludes only the Voltaire names. A later change could restrict both to generic-editor-v1/v2. This is out of scope here.
  3. comparison_instructions_test.exs, test "no publication path admits a comparison instruction": it checks only coherent?/1. A case where Configurations.create_version refuses a comparison instruction (:editor_profile_mismatch) would cover the binding path too.

Verdict: ready to merge at f90494d. Nothing REQUIRED.

Executor's disposition: RECOMMENDED 1 applied as e011ca6 (one line of the Bierce summary), confirmed separately below. RECOMMENDED 2 and 3 predate this PR or widen its scope; recorded as follow-ups.

@holden

holden commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Independent exact-head review of e011ca6

4 November 2026, by a second fresh subagent of the #307 Stage 1 executor that had not seen the author's reasoning; posted verbatim by the executor. It covers the one commit after the reviewed f90494d.

What I checked

  • git rev-parse HEAD gives e011ca6f21862940e3a24028ae53e041641c52ea, the expected head. The working tree is clean.
  • git diff f90494d e011ca6 covers one commit and one file, priv/curation/agents/instructions/bierce-whole-entry-v1.md, with one line changed (line 6). The new wording, "an exact, economical contrast over a loud one", now says what dossier.selection says. That fixes the "quiet" drift. The longest word run the new line shares with the dossier is 4 words.
  • The 8-word rule: I dumped Packet.instruction!("bierce-whole-entry-v1") with mix run --no-start --no-compile in _s1e (the loaded beam is newer than packet.ex and the dossier JSON). Checked with an Elixir script using ~r/[^\p{L}\p{N}_]+/u on lowercased text: the instruction has 853 words and the summary 191; there are 0 eight-word overlaps, and the summary still starts with "# ".
  • git grep at e011ca6 over lib/, test/ and config/ for agents/instructions, instructions/ and the filename: nothing matches, so no test outcome depends on this file.

REQUIRED
None.

RECOMMENDED
None. For information: the longest overlap with the instruction is 7 words, on line 7 (unchanged by this commit); one more matching word there would break the 8-word rule.

Verdict: GO. e011ca6 is ready to open as a PR.

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

Caution

Review failed

The pull request is closed.

Walkthrough

The runtime adds comparison-only Hitchens and Bierce instructions. Each combines the grounded version 7 policy with its dossier lens. The comparison registry and runbook now include both arms.

Changes: Persona comparison instructions

Layer / File(s) Summary
Compile persona instructions
lib/devils_dictionary/curation/runtime/packet.ex, priv/curation/agents/instructions/*, test/devils_dictionary/curation/agents/persona_dossiers_test.exs, test/devils_dictionary/curation/runtime/comparison_instructions_test.exs
The runtime compiles Hitchens and Bierce instructions from their dossier lenses and the grounded version 7 policy. Tests check instruction composition, digests, and dossier text.
Enable comparison arms
lib/devils_dictionary/curation/whole_entry.ex, lib/devils_dictionary/curation/runtime/packet.ex, test/devils_dictionary/curation/runtime/*, docs/curation/runtime-operations.md
The comparison registry and packet instruction map include both arms. Tests check runtime allowlisting, parity, and instruction versions. The runbook documents model setup and a four-arm dry run.

Estimated code review effort: 2 (Simple) | ~12 minutes

Possibly related issues

  • Issue 193: The PR adds Hitchens and Bierce instructions for the persona comparison programme.
  • Issue 298: The PR adds both arms to the runtime allowlist for non-publishing comparisons.
  • Issue 305: The PR adds the Hitchens and Bierce instructions named for the four-arm Stage 1 comparison.
  • Issue 307: The PR adds pinned Hitchens and Bierce instructions for a non-publishing Stage 1 run.

Possibly related PRs

Merge Risk: Minimal · up to e011c

The new persona arms are comparison-only. No merge-blocking issue is established.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant